Feature: Book Genie Mode - Automated Testing with Detailed Reporting
File name: BookGenie.feature
Relative path: features/BookGenie.feature
Scenario Outline: Book Genie Response Validation with Citation Verification
When
I open the Creative Workspace login page
00:00:01.247
Then
I should see the homepage
00:00:00.035
When
I click on the mode selection dropdown
00:00:02.868
locator.waitFor: Error: strict mode violation: getByText('BookGenieQA') resolved to 2 elements:
1) (p)Loading BookGenieQA(/p) aka getByText('Loading BookGenieQA')
2) (span class="text-base")BookGenieQA(/span) aka getByText('BookGenieQA', { exact: true })
Call log:
- waiting for getByText('BookGenieQA') to be visible
at BookGeniePage.(anonymous) (/home/runner/work/HC_Epub_Automation/HC_Epub_Automation/tests/pages/BookGeniePage.ts:36:41)
at /home/runner/work/HC_Epub_Automation/HC_Epub_Automation/tests/pages/BookGeniePage.ts:41:71
at __awaiter (/home/runner/work/HC_Epub_Automation/HC_Epub_Automation/tests/pages/BookGeniePage.ts:37:12)
at BookGeniePage.checkModeVisiblity (/home/runner/work/HC_Epub_Automation/HC_Epub_Automation/tests/pages/BookGeniePage.ts:72:16)
at CustomWorld.(anonymous) (/home/runner/work/HC_Epub_Automation/HC_Epub_Automation/tests/steps/bookGenie.step.ts:16:30)
at /home/runner/work/HC_Epub_Automation/HC_Epub_Automation/tests/steps/bookGenie.step.ts:8:71
at __awaiter (/home/runner/work/HC_Epub_Automation/HC_Epub_Automation/tests/steps/bookGenie.step.ts:4:12)
at CustomWorld.(anonymous) (/home/runner/work/HC_Epub_Automation/HC_Epub_Automation/tests/steps/bookGenie.step.ts:24:12)
And
I select the "BookGenieQA" mode
0s
Then
I type "Suggest 5 books on christmas" on chat input element
0s
And
I wait for AI to complete thinking
0s
And
I validate the response is visible for "Suggest 5 books on christmas"
0s
And
I extract book data from BookGenie response
0s
And
I save BookGenie book data to Excel file for query "Suggest 5 books on christmas"
0s
And
I validate each book individually against Excel file for query "Suggest 5 books on christmas"
0s
And
I validate that reason texts match citation texts with 80% similarity
0s
And
I generate detailed citation validation report
0s
Then
I verify database connectivity
0s
And
database should contain 10032 books
0s
And
I validate extracted books individually against database
0s
And
I validate BookGenie response relevance with AI for query "Suggest 5 books on christmas"
0s
Then
I should see book cards
0s
And
I extract detailed content from the book card
0s
And
I extract expected data from the left-side chat panel
0s
And
I validate book titles of all cards against chat data
0s
And
I validate authors of all cards against chat data
0s
And
I validate relevance scores of all cards against chat data
0s
And
I validate why match reasons of all cards against chat data
0s
And
I validate all card contents comprehensively against chat data
0s
=== SCENARIO LOGS: Book Genie Response Validation with Citation Verification === [11:16:47 AM] Navigating to Creative Workspace login page [11:16:49 AM] Homepage visibility: false [11:16:49 AM] ℹ️ Clicking on mode selection dropdown [11:16:52 AM] ✅ Mode selection dropdown clicked successfully [11:16:52 AM] ℹ️ Checking visibility of mode: BookGenieQA === END LOGS ===
Scenario Logs: Book Genie Response Validation with Citation Verification
[11:16:47 AM] Navigating to Creative Workspace login page
[11:16:49 AM] Homepage visibility: false
[11:16:49 AM] ℹ️ Clicking on mode selection dropdown
[11:16:52 AM] ✅ Mode selection dropdown clicked successfully
[11:16:52 AM] ℹ️ Checking visibility of mode: BookGenieQA
Scenario Outline: Book Genie Response Validation with Citation Verification
When
I open the Creative Workspace login page
00:00:00.964
Then
I should see the homepage
00:00:00.033
When
I click on the mode selection dropdown
00:00:02.650
Then
I can see the "BookGenieQA" mode
00:00:00.021
And
I select the "BookGenieQA" mode
00:00:04.825
Then
I type "Suggest 3 books for childrens" on chat input element
00:00:31.080
And
I wait for AI to complete thinking
00:00:50.521
And
I validate the response is visible for "Suggest 3 books for childrens"
00:00:00.007
And
I extract book data from BookGenie response
00:00:00.008
And
I save BookGenie book data to Excel file for query "Suggest 3 books for childrens"
00:00:00.000
+ Show Error
Error: No book data extracted from BookGenie response.
at CustomWorld.(anonymous) (/home/runner/work/HC_Epub_Automation/HC_Epub_Automation/tests/steps/bookGenie.step.ts:46:15)
at /home/runner/work/HC_Epub_Automation/HC_Epub_Automation/tests/steps/bookGenie.step.ts:8:71
at __awaiter (/home/runner/work/HC_Epub_Automation/HC_Epub_Automation/tests/steps/bookGenie.step.ts:4:12)
at CustomWorld.(anonymous) (/home/runner/work/HC_Epub_Automation/HC_Epub_Automation/tests/steps/bookGenie.step.ts:59:12)
And
I validate each book individually against Excel file for query "Suggest 3 books for childrens"
0s
And
I validate that reason texts match citation texts with 80% similarity
0s
And
I generate detailed citation validation report
0s
Then
I verify database connectivity
0s
And
database should contain 10032 books
0s
And
I validate extracted books individually against database
0s
And
I validate BookGenie response relevance with AI for query "Suggest 3 books for childrens"
0s
Then
I should see book cards
0s
And
I extract detailed content from the book card
0s
And
I extract expected data from the left-side chat panel
0s
And
I validate book titles of all cards against chat data
0s
And
I validate authors of all cards against chat data
0s
And
I validate relevance scores of all cards against chat data
0s
And
I validate why match reasons of all cards against chat data
0s
And
I validate all card contents comprehensively against chat data
0s
=== SCENARIO LOGS: Book Genie Response Validation with Citation Verification === [11:16:53 AM] Navigating to Creative Workspace login page [11:16:54 AM] Homepage visibility: false [11:16:54 AM] ℹ️ Clicking on mode selection dropdown [11:16:56 AM] ✅ Mode selection dropdown clicked successfully [11:16:56 AM] ℹ️ Checking visibility of mode: BookGenieQA [11:16:57 AM] ✅ Mode "BookGenieQA" is visible [11:16:57 AM] ℹ️ Selecting mode: BookGenieQA [11:16:57 AM] ✅ Mode "BookGenieQA" selected successfully [11:16:59 AM] ℹ️ Waiting for mode transition to complete [11:17:01 AM] ℹ️ Typing query: "Suggest 3 books for childrens" [11:17:31 AM] ⚠ ⚠Book Genie mode did not loaded [11:17:31 AM] ℹ️ Preparing to type query: "Suggest 3 books for childrens" [11:17:31 AM] ℹ️ Waiting for chat input to be visible [11:17:31 AM] ℹ️ Clearing chat input [11:17:31 AM] ℹ️ Typing query: "Suggest 3 books for childrens" [11:17:32 AM] ℹ️ Pressing Enter to submit query [11:17:32 AM] ✅ Query submitted successfully: "Suggest 3 books for childrens" [11:17:32 AM] ℹ️ Waiting for AI to complete thinking... [11:17:32 AM] 📋 WAITING FOR AI RESPONSE [11:17:32 AM] ℹ️ Looking for AI thinking indicator... [11:17:32 AM] ℹ️ Waiting for thinking indicator to appear (max 2 minutes) [11:17:34 AM] ✅ ✓ AI thinking indicator appeared - AI is processing the request [11:17:34 AM] ℹ️ Waiting for thinking indicator to disappear (max 5 minutes) [11:18:20 AM] ✅ ✓ AI thinking completed - Response should be ready [11:18:20 AM] ℹ️ 🔍 Checking for "None of the above, just" option... [11:18:20 AM] ℹ️ ✓ "None of the above, just" option not present - proceeding normally [11:18:20 AM] ℹ️ Allowing additional time for response rendering [11:18:23 AM] ✅ AI response wait process completed [11:18:23 AM] ℹ️ Validating response for query: "Suggest 3 books for childrens" [11:18:23 AM] 📋 VALIDATING RESPONSE FOR QUERY: "Suggest 3 books for childrens" [11:18:23 AM] ℹ️ Locating the latest response in chat [11:18:23 AM] ℹ️ Waiting for response to be visible (max 30 seconds) [11:18:23 AM] ✅ Response received - Length: 3006 characters [11:18:23 AM] ℹ️ Response preview: Question Type: Book Metadata — AnalyzedWatch Me WorkLet me connect the dots for you...Retrieving Re... [11:18:23 AM] ℹ️ Extracting book data from response... [11:18:23 AM] 📋 EXTRACTING BOOK DATA FROM RESPONSE [11:18:23 AM] ℹ️ Locating BookGenie response container [11:18:23 AM] ℹ️ Extracting HTML content from response [11:18:23 AM] ℹ️ HTML content extracted - Length: 32250 characters [11:18:23 AM] ℹ️ Parsing HTML to extract book data [11:18:23 AM] ✅ SUCCESSFULLY EXTRACTED 0 BOOKS [11:18:23 AM] 📋 EXTRACTED BOOKS SUMMARY [11:18:23 AM] 📊 Total books extracted: 0 [11:18:23 AM] 📊 Average relevance score: 0% [11:18:23 AM] ✅ Extracted 0 books from response === END LOGS ===
Scenario Logs: Book Genie Response Validation with Citation Verification
[11:16:53 AM] Navigating to Creative Workspace login page
[11:16:54 AM] Homepage visibility: false
[11:16:54 AM] ℹ️ Clicking on mode selection dropdown
[11:16:56 AM] ✅ Mode selection dropdown clicked successfully
[11:16:56 AM] ℹ️ Checking visibility of mode: BookGenieQA
[11:16:57 AM] ✅ Mode "BookGenieQA" is visible
[11:16:57 AM] ℹ️ Selecting mode: BookGenieQA
[11:16:57 AM] ✅ Mode "BookGenieQA" selected successfully
[11:16:59 AM] ℹ️ Waiting for mode transition to complete
[11:17:01 AM] ℹ️ Typing query: "Suggest 3 books for childrens"
[11:17:31 AM] ⚠ ⚠Book Genie mode did not loaded
[11:17:31 AM] ℹ️ Preparing to type query: "Suggest 3 books for childrens"
[11:17:31 AM] ℹ️ Waiting for chat input to be visible
[11:17:31 AM] ℹ️ Clearing chat input
[11:17:31 AM] ℹ️ Typing query: "Suggest 3 books for childrens"
[11:17:32 AM] ℹ️ Pressing Enter to submit query
[11:17:32 AM] ✅ Query submitted successfully: "Suggest 3 books for childrens"
[11:17:32 AM] ℹ️ Waiting for AI to complete thinking...
[11:17:32 AM] 📋 WAITING FOR AI RESPONSE
[11:17:32 AM] ℹ️ Looking for AI thinking indicator...
[11:17:32 AM] ℹ️ Waiting for thinking indicator to appear (max 2 minutes)
[11:17:34 AM] ✅ ✓ AI thinking indicator appeared - AI is processing the request
[11:17:34 AM] ℹ️ Waiting for thinking indicator to disappear (max 5 minutes)
[11:18:20 AM] ✅ ✓ AI thinking completed - Response should be ready
[11:18:20 AM] ℹ️ 🔍 Checking for "None of the above, just" option...
[11:18:20 AM] ℹ️ ✓ "None of the above, just" option not present - proceeding normally
[11:18:20 AM] ℹ️ Allowing additional time for response rendering
[11:18:23 AM] ✅ AI response wait process completed
[11:18:23 AM] ℹ️ Validating response for query: "Suggest 3 books for childrens"
[11:18:23 AM] 📋 VALIDATING RESPONSE FOR QUERY: "Suggest 3 books for childrens"
[11:18:23 AM] ℹ️ Locating the latest response in chat
[11:18:23 AM] ℹ️ Waiting for response to be visible (max 30 seconds)
[11:18:23 AM] ✅ Response received - Length: 3006 characters
[11:18:23 AM] ℹ️ Response preview: Question Type: Book Metadata — AnalyzedWatch Me WorkLet me connect the dots for you...Retrieving Re...
[11:18:23 AM] ℹ️ Extracting book data from response...
[11:18:23 AM] 📋 EXTRACTING BOOK DATA FROM RESPONSE
[11:18:23 AM] ℹ️ Locating BookGenie response container
[11:18:23 AM] ℹ️ Extracting HTML content from response
[11:18:23 AM] ℹ️ HTML content extracted - Length: 32250 characters
[11:18:23 AM] ℹ️ Parsing HTML to extract book data
[11:18:23 AM] ✅ SUCCESSFULLY EXTRACTED 0 BOOKS
[11:18:23 AM] 📋 EXTRACTED BOOKS SUMMARY
[11:18:23 AM] 📊 Total books extracted: 0
[11:18:23 AM] 📊 Average relevance score: 0%
[11:18:23 AM] ✅ Extracted 0 books from response